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Working in triads: A case study of a peer review process 


Abstract 

Peer review of teaching has become an accepted educational procedure in Australia to quality assure the 
quality of teaching practices. The institutional implementation of the peer review process can be viewed as 
genuine desire to improve teaching quality or an imposition from above as a measure of accountability and 
performativity. One approach is to conduct the peer review process as a team or a triad; involving a group of 
three academics. This article reviews this process of peer review through the eyes of the participants. The 
results of the study indicate that the peer review process upon which this study is based; has the potential to 
not only significantly impact academics’ pedagogy but to improve teaching confidence and associated benefits 
in regard to evidence based teaching for promotional opportunities. 
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Introduction 

Tertiary providers are increasingly being encouraged to be more accountable, as 
shown by the formation of various bodies and entities devoted to assuring the quality 
of research, standards and university teaching. The Bradbury report (DEEWR, 2008) 
resulted in the Government setting up a new agency called the Tertiary Education 
Quality and Standards Authority, or TEQSA. More recently there has been a focus on 
threshold standards, some of which have been outlined in the Higher Education 
Standards Framework (Department of Industry, Innovation, Science, Research and 
Tertiary Education, 2011). 

This is in keeping with national and international attention to academic standards in 
research quality (e.g. OECD, 2011), and increasing accountability for quality (e.g. the 
UKs HEFCE’s Research Excellence Framework and Australia’s ‘Excellence for 
Research in Australia’). There is an increasing requirement to demonstrate appropriate 
standards to professional accrediting agencies and quality-audit agencies such as 
TEQSA. The quality-enhancement and quality-assurance agendas are very important to 
universities in Australia and internationally as a competitive benchmark for attracting 
students and as a means of anticipating the TEQSA standards on teaching. 

Related to the accountability agenda is the increased focus on peer review of teaching 
(PRoT) processes in tertiary institutions. PRoT processes used in Australian 
universities differ significantly across institutions, in terms of protocols, purposes and 
instruments used to gather data. These elements are a reflection of each institution’s 
organisational setting and policy and process context that governs PRoT and 
observation. PRoT’s purposes range from purely developmental purposes to 
promotion purposes (Crisp, 2010) or a combination of both; the relative importance of 
each is governed by the attitudes of the university’s executive about valid data and 
evidence to support applications for promotions, awards and grants. 

The protocols, puiposes and instruments (criteria) used to gather data in relation to the 
PRoT processes have implications for the organisation and the individual. Each 
criterion also has linkages to other criteria that moderate its effect. For example, some 
institutions are interested in formative PRoT processes aimed at sessional staff, new 
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academics or all staff. Others are focused on summative forms for promotion, and 
align these with developmental models so that development in the university can be 
aimed at internal standards and assessed by external observers as part of the 
accountability agenda. 

The puipose of this paper is twofold: to review the PRoT tertiary landscape, including 
systems/protocols/models with their stated aims and purposes, and to illuminate this 
discussion via a study of participant views into the peer review process operating at a 
single tertiary institution. 

The first focus will be achieved through a review of a cross-section of models and 
protocols derived from an audit of universities’ public websites, which will enable the 
reader to make decisions about designing their own model. This review will consider 
the organisational setting and the PRoT purpose, process (that the observer/observee 
experiences) and protocol (number of observers, number of sessions, etc.), and the 
instruments that are the differentiators in most cases. Strengths and weaknesses in the 
various models will be identified using experiential reflections from the authors, 
based on case studies from two universities that use PRoT processes at either end of a 
continuum that ranges from passive encouragement to controlled compliance. 

The second focus will be achieved by reporting a case study of peer-review processes 
operating at a university in Australia. This case study was followed by a post-review 
process during which each of the participants reflected on their experiences of the 
peer-review protocols. These reflections were designed to give evidence of the 
process from a different perspective: that of the participants themselves. The 
reflection was structured according to a framework: reasons for participating; initial 
reactions prior to review; reactions during review; and impact. The authors 
participated first-hand in these processes as part of a single triad, while employed as 
staff members at the tertiary institution. A review of the literature did not reveal any 
study that has reported participation in the process from a participant’s perspective; 
this is the distinctive aspect of this study. The key research questions were: 

1. How different or similar are the PRoT processes operating in Australian 
universities? 
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2. What are the various contexts that influence the nature of each of these PRoT 
processes? 

3. What are the advantages and disadvantages of each of the PRoT systems in 
operation? 

4. How do participants feel about participation in peer-review processes? 

5. How can decisions be made about which PRoT system is best for a given 
institution? 

The Role of Peer Review in Higher Education 

Significant and visible accountability agendas are now commonplace all over the 
world as a result of the economisation of educational policy; this is further shown by 
the introduction of performance indicators to increasingly scrutinise the work of 
academics (Smyth, Dow, Hattman, Reid & Shacklock, 2000). In Australia, the 
creation of websites such as My University provide further evidence of a culture of 
performance measurement that provides a mechanism for the wider community to 
compare the performance of tertiary institutions. Peer review of teaching can be 
considered as a quality-assurance measure for evaluating the quality of academic 
teaching at a given university. 

PRoT can be defined as a formative or summative internal process in which teaching 
is reviewed and scrutinised by colleagues, or in summative cases by external 
reviewers (Crisp, 2010), in an attempt to provide a better product for students in the 
form of better teaching and a heightened student experience. According to Harris, 
Farrell, Bell, Devlin and James (2008/9, p.5), PRoT is “academic colleagues giving 
and receiving feedback on their teaching practices and its effectiveness in promoting 
student learning”. The PRoT process can be used for developmental purposes or to 
collect and collate evidence to support application for promotion or for grant and 
award applications. 

Quality university teaching has been a major focus for universities, and the 
dimensions of effective teaching have been defined in many studies in an attempt to 
promote conditions that would enhance the student experience in universities. In 1994 
Marsh and Roche developed the Student Evaluation of Educational Quality (SEEQ) in 
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an attempt to articulate specific characteristics of university teachers. They identified 
nine dimensions: 

(1) Learning/academic value; 

(2) Lecturer enthusiasm; 

(3) Organisation and clarity; 

(4) Group interaction; 

(5) Individual rapport; 

(6) Breadth of coverage; 

(7) Examinations/grading; 

(8) Assignment/reading; and 

(9) Workload/difficulty. 

Devlin and Samarawickrema (2010) compared these nine dimensions with the 
Australian Learning and Teaching Council’s (ALTC) five key guiding criteria for 
determining excellence in university teaching for the purposes of recognition and 
reward: 

(1) Approaches to teaching that influence, motivate and inspire students to learn; 

(2) Development of curricula and resources that reflect a command of the field; 

(3) Approaches to assessment and feedback that foster independent learning; 

(4) Respect and support for the development of students as individuals; and 

(5) Scholarly activities that have influenced and enhanced learning and teaching 
(ALTC, 2008). 

The authors found a strong relationship between the ALTC criteria and the validated 
dimensions of effective teaching from the peer-reviewed literature, and concluded, “In 
effect, the ALTC criteria have become accepted as the proxy list of skills and 
practices of effective university teaching in Australian higher education” (Devlin & 
Samarawickrema, 2010, p. 115). 

In addition to face-to-face teaching, PRoT can also refer to review of courses, 
assessment practices, online learning environments, curriculum design and resource 
development. A review of the literature reveals several models of teaching review. 
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Gosling (2002, p.84) identified the evaluation model (performance management), the 
developmental model (improvement) and the PRoT model (reflective). 

There have been a number of major projects reporting on peer review in tertiary 
institutions in Australia. One PRoT project ran from July 2007 to December 2008, 
and was funded under the 2007 round of the Australian Learning and Teaching 
Council (ALTC) Grants Scheme - Priority Projects Program. In that project’s review 
of the literature, some forms of PRoT were identified as early as the late 1990s and 
the early years of the new millennium. The outcome of the 2007 project was a 
handbook that supported institutions in developing and embedding effective PRoT 
policies and practices (Harris, Farrell, Bell, Devlin & James, 2008). 

The handbook reports that PRoT was not part of university policy or culture in 
Australia. Further, there was no record of PRoT being used formally in Australian 
tertiary institutions nor as a widespread professional activity and hence a reasonable 
assumption is that relatively few academic staff at Australian universities have 
engaged in PRoT in any systematic or formalised way (Harris, et al., 2008, p.10). 

They found that although in 50% of the universities reviewed (13/26), PRoT was an 
available source of evidence for promotion, exactly how many academics used it was 
not clear. The review found that the main obstacles were negative perceptions of the 
mandated nature of PRoT and a widely held closed door teaching philosophy that 
viewed PRoT as an impost and merely a measure of accountability (Harris et al., 
2008). 

A review of the PRoT literature highlights several points: the importance of voluntary 
participation in a process that is not linked to performance management by an external 
party; the significant learning taking place for the observer as they review their 
colleagues’ teaching performance; the importance of conducting the PRoT process as 
a triad team (supportive peer, teacher being observed; external independent 
expert/developer); and repeat sessions involving members of the triad (Bolt & 
Atkinson, 2010, p.89). In common is the cyclical approach involving four stages: pre¬ 
observation, discussion, observation and feedback/reflection. 
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PRoT has also been referred to as a process of quality enhancement rather than a 
quality-assurance instrument. This view is held by a number of authors, including 
Wilkins and Shin (2011), Farrell (2011) and Hargreaves (2000), who noted that peer 
review, is primarily a means to promote professional development, collaboration and 
self-assessment, and results in increased collegiality amongst staff by enhancing 
communication between colleagues. Lomasa and Nicholls (2005) claim that peer 
review is a quality-enhancement tool that assists academics in scrutinising their 
teaching, culminating in enhanced student learning as a result of self-improvement 
and transference of information from the peer review. 

Blackmore (2005) added a further dimension, noting that peer review has been used 
not only to improve teaching but also to assist in the decision-making processes for 
promotion and salary increases. On the other hand, Lomasa and Nicholls (2005) 
identified PRoT as an imposition by external agencies because the word “review” had 
been associated with the inspectorial version used for quality assurance rather than 
quality enhancement. This is reinforced by the views of Lomasa and Nicholls (2005), 
who noted an emerging view that the rationale for peer review can vary along a 
continuum from an agenda of performance-management to managerialism, 
monitoring and surveillance. This may be one reason why academics have not 
traditionally participated in peer review voluntarily. 

The Peer-Review Landscape in Australia 

PRoT models at 20 universities in Australia were reviewed via the public information 
available on each university’s website. This public information was organised into 
approximately 30 headings, or characteristics, used as a basis for this review. These 
characteristics are related to the framework outlined in the ALTC handbook (Harris, 
et al., 2009) and are used as a system for analysis of this public information. The 
seven questions are: 

1. Whose teaching will be reviewed? 

2. What will be the policy regarding participation? 

3. What will be reviewed? 

4. Who will the reviewers be? 
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5. What form will the review process take? 

6. What reporting will take place? 

7. What type of follow-up will occur after completion of the PRoT process? 

Analysis of the publicly available data from university websites resulted in the 
development of a tabular collation indicating various aspects, issues and 
characteristics of the various PRoT processes at work related to these seven questions.. 
For example, a common characteristic was statements indicating the voluntary or 
compulsory status of the PRoT process operating at the institution. The seven 
questions capture how the PRoT systems operate within the organisations reviewed, 
and have implications for the design criteria related to the implementation of PRoT in 
Australian universities. 

Question 1: Whose teaching will be reviewed? 

This is basically a question about limiting or opening the PRoT program. In all cases 
except one, PRoT was made available to all academics, including sessional staff, as a 
developmental process, a summative process or both. Many institutions in Australia 
rely on sessional staff to teach their programs. New accountability agendas 
(Department of Industry, Innovation, Science, Research and Tertiary Education, 

2011) will serve to focus more attention on suitably qualified staff to teach in 
Australian tertiary institutions, including sessional staff and other outsourcing avenues. 
At one university, for example, currently 51 % of face-to-face teaching is executed by 
sessional staff, so this criterion ensures that they are well trained and have access to 
quality-enhancement processes. The decision not to mandate this for all staff relates 
to the fact that sessional staff are paid on a casual basis, so getting sessional staff to 
take part in PRoT that has some time component has a significant dollar value 
attached. Some universities have made this a condition of employment to be 
undertaken gratis, but that lowers the value statement for sessional staff and is clearly 
a source of tension. 

According to the review undertaken for this paper, the range of possible participant 
groups are full-time academics, sessional staff and contract staff. Most academics 
have an existing job description that does not include PRoT or observation; thus 
PRoT must be fitted into existing workloads and expectations. Contract staff can be 
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given conditions that require them to undertake some quality enhancement, but this 
may make the engagement in PRoT less appealing. Then there is the aspect of 
limiting it to certain people such as those new to teaching, or to a particular course or 
to the university. It may be made a condition of probation/confirmation of 
appointment. It may be applied to levels A, B and C, but not D and E, for various 
reasons. It may be limited to staff that have demonstrated need of assistance as 
reflected in Student Evaluation of Teaching and Student Evaluation of Course data or 
some other metric. 

Question 2: What will be the policy regarding participation? 

Most PRoT processes were described as purely voluntary, although some were tied to 
promotional opportunities and conditions of probation for new and early-career 
academics. Only one institution made PRoT compulsory. Almost all universities 
strongly encouraged participation for all staff in some form of PRoT process. 

The decision regarding voluntary versus compulsory participation relates in some 
ways to the conditions of employment. At another level, it is about creating a system 
that expects that all staff will engage with the process to meet progression and 
promotion requirements. The decision regarding participation relates to 
considerations of negotiation with the relevant union to create an enterprise 
bargaining agreement that has, in some sense, “de-toothed” the system to create 
something quite innocuous, with minimal extra engagement or added workload, such 
as exists at some of the universities reviewed. Having a system of voluntary PRoT is 
philosophically ideal, but implies limitations on what the system can expect 
participants to do. Participants may be intrinsically motivated, but that intrinsic 
interest may be in their own progression rather than in teaching or PRoT per se, so 
their participation is never fully guaranteed. Some institutions have recognised a 
clear understanding of academic motivators, such that there are tangible advantages to 
taking part in PRoT. Some of these motivators include things such as credit towards 
some kind of recognised certificate, evidence towards awards and promotions, 
assistance in meeting promotion requirements and data that can be used to write case 
studies for publication. 
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Many of the universities reviewed had put a website together that functions as a “pull” 
medium; that is, academics are expected to engage with it of their own accord. 
However, without active engagement or sufficient inducement, academics may not 
engage. This is a passive approach. Some universities have a “push” medium that 
sends academics links to the site and to various resources regularly via email or social 
media. Again, this passive approach is unlikely to gather momentum. A website 
needs academics’ active engagement with the resources provided so they know how 
to apply them using experiential learning. This is what can be referred to as a 
customer-focused model. If intrinsic interest is not enough, strategic interest needs to 
be piqued by supplying policies and provisions that make engagement necessary for 
future work or promotion. In this way quality of involvement is maintained at a high 
level in a voluntary system. With compulsory participation there also needs to be a 
statement of quality of participation so that it is not done in a cursory manner. Again, 
it depends on the commitment of the organisation to the process and the value 
statement that is assigned to peer observation or review. 

Question 3: What will be reviewed? 

A wide variety of PRoT processes were identified, ranging from processes designed 
purely for promotional purposes and those designed solely for developmental or 
formative purposes. Most institutions included teaching both face-to-face and remote 
teaching (including online delivery) of PRoT for course materials, assessment design 
and marking and other aspects related to teaching such as course evaluation and 
renewal. 

In reviewing the institutions, PRoT included face-to-face lectures, but could be 
extended to tutorials, workshops, clinical practice, 1-2-1 teaching and supervision, 
studio teaching, online teaching, distance education and blended learning. In most 
cases this decision was left to the reviewee. This choice aspect ensured that materials 
to be used for review purposes were a very general set of observation criteria that 
could be easily interpreted for different learning and teaching activities, or which 
might require different protocols and specific instruments. 

Question 4: Who will the reviewers be? 
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The PRoT participants consisted mainly of pairs of academics from the same 
discipline, although some universities mandated or recommended experts from 
outside of the discipline; the two types of partners were commonly referred to as 
discipline experts (internal) and learning and teaching experts (external). There were 
some differences in terms of who the PRoT team would be; these differences clearly 
linked to the nature of the PRoT process operating at the institution. For example, one 
university insisted for any promotional process that reviewers must include academics 
from outside of the reviewee’s circle, and that these academics must be at an 
academic level at least equivalent to but mostly above that of the reviewee. In most 
cases this information was not made clear in the public information reviewed. Only 
one institution mandated a triad of participants in a reciprocal arrangement that 
ensured that each member of the triad played multiple roles as reviewer and reviewee. 
In this case, the triad consisted of two participants from the same discipline and one 
participant who came from outside of the discipline but was recognised as an expert in 
pedagogy. 

The question of reviewers relates to the definition of peers. For example, are peer 
reviewers chosen from the same seniority level (A-E), from different seniority levels, 
from the same disciplines, from different disciplines, from line management, from 
colleagues, from a pool of strangers; should there be one reviewer or multiple 
reviewers? Each has a different set of implications for what is brought to the 
observation in terms of knowledge, experience, expectation, feedback skills and 
tendency to give honest appraisals rather than being too critical or too glowing. 

Question 5: What form will the review process take? 

Most universities insisted upon one observation but strongly encouraged multiple 
observations. Most of the universities had checklists of learning objectives for 
observers to follow, many linked to the specific institution's learning and teaching 
priorities. In the main, PRoT is a four-stage process. During a pre-review phase, the 
reviewer and reviewee meet and discuss the objectives of the session to be observed. 
This is followed by the review itself, a post-review session between reviewer and 
reviewee and a final follow-through stage, characterised by an agreed action plan to 
be implemented in the future. There were many variations in the degree to which all 
the stages were required, ranging from optional, to mandated, shared and private. One 
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university offered a further stage some six months after the review, with detailed 
qualitative and quantitative data in the form of reports that were forwarded to the 
reviewee. The focus of the review itself was on the academic rather than the students. 
Only one university involved the students directly in the review, matching the student 
perceptions of the lesson to the intended objectives identified by the academic being 
reviewed. The tools used by the reviewers were mostly Likert-scale type pro-formas 
with some free-text areas. All universities identified strengths and weaknesses in 
some form. Many of the institutions reviewed did not provide data on whether the 
reviewer had an input into the pre-review planning stage. Three institutions offered or 
encouraged this. 

The form of the review process primarily concerns the protocol design and underlying 
needs of the organisation, its policy and the context and content of any 
reporting. Depending upon the university, the process reflects a developmental or a 
participatory approach related to data collection, and/ or box-ticking. Some 
institutions offered hand-written notes from the observation; others offered a truly 
analysed piece of work plotted against purposely designed criteria that could be 
verified against a framework to ascertain aspects of quality. In short, the data- 
gathering tools ranged from analysis that triangulated data from a range of 
stakeholders to data gathered from one peer’s perspective. 

Question 6: What reporting will take place? 

There was a great variety of reporting possibilities, but most universities linked 
reporting to some set of institutional goals or objectives. Some of these pro-formas 
were mandated, others strongly encouraged. On some occasions academics were 
totally free to select their own goals or objectives for the lesson. In most cases some 
kind of scaled mechanism was used, based on a type of Likert scale, identifying how 
well the academic had achieved each of the specific objectives. Free-text boxes were 
also made available in some instances. In one instance the final report was sent 
independently to the university teaching and learning group without consultation with 
the reviewee. In all cases the feedback environment was encouraged to be 
developmental and formative. However, the widespread use of Likert scales to report 
performance might suggest that this characteristic was not as supportive as the 
literature suggested. Only one institution offered a fully analysed, qualitative and 
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quantitative report triangulated to student data from the same teaching event. It 
appears that for the most part, the PRoT process ends with the first review and one 
report. 

Question 7: What type of follow-up will occur after completion of the PRoT 
process? 

Most universities at least recommended an ongoing process of PRoT to implement the 
action plan developed by the reviewee at the end of the review, although few 
mandated it. 

Follow-up on PRoT can only occur if there are multiple observation sessions, as there 
can be assistance and further review of any development ideas. If the institutional 
focus is on ticking boxes, it appears that less can be gained from participation for the 
protagonists and the organisation. It is important to identify the basic reasons for 
initiating PRoT or observation of teaching: to improve casual staff, to improve 
academic teachers or to create an atmosphere conducive to development. One 
university made a distinction between "review" and "observation" to eliminate the 
judgmental nature of “review” so that staff are more likely to be engaged and less 
likely to be discouraged. 

Miscellaneous 

Some institutions offered training for participants in the PRoT process; others tied this 
training to recognition in the form of credit towards some official certificate of 
university teaching-development through peer observation that could lead to further 
development. This adds value both to the individual and to the organisation that is 
chasing evidence of its quality processes, such as in the case of external review by 
TEQSA. Many did not specify whether this training was compulsory or not. 

The PRoT Experience from the Participants’ Perspectives 

The authors of this study participated in a peer-review process at a tertiary institution 
in Australia. Their experiences and reflections were noted after the experiences were 
completed. The reflections that follow are experiential summaries of the PRoT 
process at University A as identified by the authors. The university is a large multi- 
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campus public university based in the south-east corner of the Australian state of 
Queensland. At this university, the peer-review process is voluntary and involves a 
triad of academics: the reviewee, a discipline expert from the same school as the 
reviewee and a teaching and learning expert who comes from a different area of the 
university. The process involves a series of three events, with each academic in the 
triad being reviewed by the other two members. There are four stages: planning, 
review, post-review reflection and a follow-up stage where data is analysed and sent 
to the reviewee. The triad works together as a team for a semester. The peer-review 
data can be used for promotional purposes. 

There are a number of stages outlined by the institution’s Institute for Higher 
Education (GIHE, 2012). The first stage is the pre-observation stage, where the triad 
exchanges objectives, learning outcomes and other focuses for review. This is 
followed by the first of two reviews of each participant in the triad. The reviews are 
followed immediately by an oral feedback session and debriefing, in which future 
focuses are identified. Each feedback phase is accompanied by a written evaluation 
from the reviewers and the students. These are shared with each participant in the 
triad and sent on to the relevant university administration. During the follow-through 
stage, final reports are received from the relevant university administration by the 
participants as reviewees, including qualitative and quantitative data analysed by the 
relevant university administration, in terms of aligning the reviewees’ goals and the 
student perceptions of these goals. Specific improvements from the first review to the 
second are identified. 

The reports consist of Likert-scale results sheets linked to various questions, 
contrasting teaching objectives with student responses to teaching to determine how 
well objectives are aligned with student experiences. In addition there is a summary of 
student feedback for each session, including student comments and a summary of peer 
observer notes, which allows a “triangulation” process to occur. Exceipts from the 
reflections of each of the participants in the triad are detailed below. 

The responses are organised according to the following framework: the nature of the 
peer-review process from a reviewer’s perspective; the nature of the peer-review 
process from a reviewee’s perspective; and the inhibiting and enabling factors that 
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contribute to the peer-review experience. The participants are identified as PI, P2 and 
P3 to protect their privacy. 


This was my first formalised PRoT experience in a tertiary teaching 
environment. I assumed it had to be face-to face so I chose that to focus on. 
Now I realise / could choose other aspects ofPRoT. I was motivated 
extrinsically because I knew I could teach well with a teacher background in 
education. Hence, I was not nervous about being observed at all - in fact this 
is what I do for a living! I knew I had to make a case for employing me as 1 
was only on contract, and hence having someone influential within the school 
watch me teach was akin to confirming that person as a possible referee in the 
future. (PI) 


I was impressed by the depth of the process, the follow-through and the 
specific quantitative data / received from the teaching and learning team. 
Having two observers was a great advantage with two sets of feedback. Not 
knowing the other academics prior to forming the triad was a success factor, 
as it formalised the whole process and ensured to some exten t that there was 
objectivity in the feedback I received. Knowing that they had gone through 
some observer training in a formalised way was also a comfort, as the other 
members of the triad were not from a school of education, and hence I was 
concerned that they may not know what they were assessing. There was a set 
of criteria against which I was assessed so I was assured that any personal 
criterion for performance could not be used when evaluating my episode. 
Having students comment on the same session was excellent and truly 
informative as they come from a different perspective to the academic 
reviewers. Their perceptions of what I was trying to do did not always match 
what my objectives actually were. (PI) 


The formed post-teaching feedback session immediately following the teaching 
episode was excellent, as 1 tend to forget quickly and focus on what is coming 
next. The...reflection phase was a surprise -1 wasn’t expecting to get a follow¬ 
up call six months after the event and a follow-up set of final quantitative and 
qualitative feedback reports, which I can use forever in gaining future 
employment and/or promotion. I retrospect, although my initial motivation 
was probably purely for summative, extrinsic reasons, I learned some things 
about my teaching that ensures the process was also developmental. (PI) 


Analysis of the reflections from each of the participants in the triad revealed three 
different motivations for participation. These reactions echo a number of issues raised 
in the literature in relation to the perception of the peer-review process. One 
participant chose to be involved as a reaction against an externally imposed system of 
accountability, so in some sense, his decision to become involved was not for 
developmental reasons. He noted: 
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It was partly in revolt against this (i.e. the stress [placed] upon student 
evaluation) that 1 became involved in the PRoT project. If student 
ratings were not to be the be-all and end-all of teaching, then maybe 
we academics could help each other develop our teaching. I am 
exasperated by [the university’s] focus upon student ratings, and want 
to stress other methods of evaluation of teach ing. (P2) 

However, his reflections reveal that the process was indeed developmental and 
educative. One indication of this was his learning how to take copious reviewer notes; 
in itself a defining characteristic of the peer-review system, but not one on which 
much emphasis is placed in the literature. The major impact for (P2) was reassurance 
that he was a good teacher, and evidence for promotional purposes, as shown by the 
following reflections: 


I was stunned at the copious notes which (P2) took. In my subsequent 
obserx’er roles I emulated him. It makes the observation much harder 
work, but gives a huge reserve of comments and examples to draw on 
later. 

The process in which you [i.e. P3] and I interacted was interesting, but 
the main effect for me was simply that it confirmed that, even being 
judged by professional standards, I am a good teacher. 

I also was encouraged overall by the observation process. 1 know now 
that my teaching is good, not just by the rather haphazard standards of 
universities, but also as judged by an expert wielding a completely 
different yardstick. It has added to my confidence, and to my ability to 
look for further improvements. (PI) 

I also know that I can add my participation onto my record of service 
rendered. 

One participated because of an intrinsic desire to experience the process: 

I participated in the process because I was interested in showing my 
involvement. As a person employed on a contract, I was motivated to 
experience as much as I could whilst in the environment. Additionally, 

I was genuinely interested in peer-review processes and how they 
worked. (PI) 

The third participant became involved due to a genuine desire to improve his teaching 
skills: 


Having 15 years of experience teaching both introductory and 
advanced organic chemistry subjects, I joined the [PRoTj process 
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with hopes of not just improving my own teaching skills but improving 
student learning of difficult topics. I’ve found it difficult get students 
to engage in dicdogue during lectures. I have always been a fairly 
“traditional” lecturer and was trained to give “chalk talks” as a PhD 
student, an ultra-traditional method of teaching organic chemistry 
that can be effective in many areas of the discipline. However, this 
method is difficult to translate to the modern “PowerPoint-based” 
classroom, and have found slide-based teaching left me wanting. (PS) 

Analysis of the participant reflections in this study revealed a variety of success 
factors, including the conclusion that the process, from the perspective of both a 
reviewer and a reviewee, is a mutually beneficial one. One participant, speaking as a 
reviewer, commented in detail: 


The fact that I was observing PS in a lecture-style amphitheatre that 
seated 300 students meant that I was sympathetic to his cause - 
engaging students in such an environment is a difficult task due to the 
very physical constraints of such a structure. I secretly wished him 
luck. My fears for PS were allayed quickly - P3 was a born teacher, 
if there was such a thing, with a commanding presence, excellent 
voice that projected to the back corners of the auditorium and a warm 
personality that ensured students did not feel threatened. (PI) 

The results of the study indicate that the peer-review process upon which this study is 
based has the potential to significantly affect academics’ pedagogy and to improve 
teaching confidence. These interpretations emerge from the following detailed 
reflections from reviewee and reviewer: 


For my first peer review I attempted to change a lecture that I had 
given many times to incorporate a mixture of slides and board work. 
In retrospect, it was perhaps not the perfect time to trial something 
new and in the end it was not particularly effective. PI and P2 were 
full of suggestions, criticisms and encouragement for how I could 
improve my teaching. Getting beyond the blow to my ego for having 
an “off day”, 1 set about getting ready for my second peer review the 
next fortnight. As it was a lecture slot immediately before we were 
due to have a quiz, I gave students a brief overview about the 
material we had covered in that cycle, then divided them into teams to 
work on problems as a group. I visited groups individually, then had 
students present answers on the board and we discussed them 
amongst the entire class. The approach seemed to work for the 
students and created interaction that I could use to guide their 
learning. PI and P2 agreed that this worked significantly better than 
my first attempt. This year, I have tried to better incorporate this 
interactive approach in teaching the same subject and I received the 
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highest teaching evaluations I have had in my career, scoring 4.7/5 
for overall teaching effectiveness. (P3) 

This changed significantly the second time I observed P3 as he put 
into practice some of the suggestions we as reviewees had made. In 
terms of reviewing colleagues, P3’s case was far more difficult. 
Although I was confident in my appraisals, I was a little concerned 
about how he would react to my “criticisms”, couched as they may 
have been. To my joy, he took it all on the chin, took notice, took stock 
and delivered a significantly better lesson as a result of the process. 

The perfect outcome! (PI) 

From a reviewer’s perspective, P2 identified the major benefit as learning from 
colleagues despite the fact that the colleagues were not from the same discipline. The 
experience of seeing a colleague make a success of a lecturing episode in a confining 
space full of 300 students was a revelation, and ensures that this excuse can no be 
longer used to justify poor pedagogy. 

As a reviewer I did not expect to come away from the process 
learning anything new, other than a confirmation of what not to do. 

This was entirely a misconception, as I learned things from P3’s 
performance that I could also put into practice, none more so than the 
irrelevance of a confining structure in which to teach. In a nutshell, if 
the students are engaged, the structure does not impact. I won’t be 
using this as an excuse anymore. (PI) 

P3’s reflections are more traditional, dealing largely with involvement in order to 
improve pedagogy. In terms of outcomes, there can be no doubt that his participation 
can be viewed as a huge success, as shown by the fact that his next effectiveness 
rating was the highest he had ever achieved in 15 years. Specifically, it resulted in a 
change in pedagogy to incorporate new approaches to engage students. 

In retrospect, lam glad that I participated in the [PRoT] project, as I 
feel it has given me insight in how to move beyond the traditional 
monologue style of teaching. However, it is easy to slip back into that 
mode, as interactive teaching requires more preparation than just 
dusting off old lecture notes I have given several times before. One 
must be able to adapt to the unique needs of each cohort and be more 
dynamic and flexible about the overall direction of an interactive 
session. In the future, I hope that 1 can provide useful insight and 
mentoring for other academics seeking to enhance their teaching 
through peer review. (P3) 

Of significance is that this participant expressed a desire to become a mentor for other 
academics and an advocate for peer-review involvement. 
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The success factors of these experiences can be identified. The key factor is a triad 
structure in which one participant is a “stranger”. This encourages honest evaluations 
of peers in a workplace environment that would be difficult to negotiate if all 
participants were colleagues in the same discipline. The presence of participants from 
outside the discipline can be viewed as a success factor because it allows the reviewer 
to focus on pedagogy and not discipline knowledge, which could otherwise 
potentially influence a reviewer and distract them from the skills being evaluated. 
Initial reactions from the participants revealed some apprehension about participating 
as strangers in unfamiliar contexts: 


As a reviewer, / was a little apprehensive about evaluating a 
colleague who was not known to me and hence / had no relationship 
to help me define my role in the process. I was unsure of what to 
expect from an academic who came from a discipline of which 1 had 
no knowledge at all. The fact that I knew little of science - in other 
words, content knowledge - weighed on my mind quite a great deal. 

(PI) 

I had no idea that PS was nervous about the reviewing process.... He 
came across as highly confident, with knowledge of teaching pivoting 
upon his expertise as an educator of schoolteachers. I had highly 
mixed feelings about that. On the one hand, intellectually I knew that 
there may be little difference between teaching in the senior levels of 
a high school and teaching first-year undergraduates at university. 

On the other hand, there is an ethos at university that teaching here is 
different. We can’t pressure students to do their homework, or even 
attend class. And the students do get to rate us in ways that would be 
unthinkable at the school level. (P2) 

The second reviewer was from the same discipline as the reviewee, and hence could 
make knowledgeable comments on the content of the teaching episode; this ensured a 
specialisation structure to the feedback, where one reviewer focused on content and 
the other on pedagogy. The triad structure provides a balance for the reviewee, in 
terms of triangulation of feedback. The triad structure mandates more than one 
review episode; this allows the reviewee to put into action any feedback from the first 
review, this ensuring a cycle of reflection leading to action and change. 


Summary and Discussion 
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Peer review of teaching has become an accepted university procedure in Australia to 
assure the quality of teaching practices in university settings, in an attempt to improve 
learning outcomes for students. A review of peer review of teaching protocols in 
Australian universities identified a variety of approaches. This review failed to 
identify any studies that focused on the process from a participant’s perspective. It is 
acknowledged that this study cannot be generalised due to the low participant 
numbers. 

Low participation rates in the process of peer review are a characteristic of Australian 
universities. The reasons for this low participation could be related to the lack of 
compulsory status; a rejection of the perceived imposition from management above; 
or a genuine disbelief in the value of participating in the peer-review process. This 
section is organised in terms of the research questions: 

1. How different or similar are the PRoT processes operating in Australian 
universities? 

2. What are the various contexts that influence the nature of each of these PRoT 
processes? 

3. What are the advantages and disadvantages of each of the PRoT systems in 
operation? 

4. How do participants feel about participation in peer-review processes? 

5. How can decisions be made about which PRoT system is best for a given 
institution? 

One of the most striking results of the review is the variety of PRoT processes 
operating in Australian universities at present, ranging from very formalised, 
structured and mandatory processes to less structured and informal systems. Most 
institutions reviewed are characterised by informal, voluntary processes with 
significant participant buy-in that is motivated intrinsically by promotional and/or 
developmental priorities; but that may lack integrity due to the casual nature of these 
processes. In all but one case, PRoT is voluntary. This disparity can be illustrated in 
the form of a continuum ranging from passive encouragement to controlled 
compliance. 
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It is the context that influences the nature of these processes. All PRoT processes have 
advantages and disadvantages (outlined in the next section). To make sense of the 
data, this paper will now identify the potential benefits and downfalls of each of the 
design characteristics for each of the stakeholders (participants and university 
administration) from a procedural and an organisational standpoint. 

Un iversity a dm ini strati on 

PRoT offers tertiary institutions an opportunity to ensure that claims made for 
excellence are demonstrable and claims against organisational (or national) standards 
can be assessed. In an ever-increasing environment of accountability, this would 
appear to be a major strength. Systematising PRoT will require some resources in 
terms of enough trained and trusted observers who are at arms’ length to the person 
being observed to maintain objectivity. For example, for the PRoT system to work 
effectively, a focused set of quality criteria for relevant learning and teaching 
activities arranged into a rubric will need to be developed. Having external observers, 
in addition to internal, may assist in providing benchmarked quality expectations. 

The PRoT system provides some degree of certainty for supervisors that promotion, 
confirmation or awards are deserved. PRoT has a distinct advantage for those 
academics whose strengths lie in teaching, as it expands the data set that is used to 
make promotion decisions beyond just publication history and grant income. It 
provides an increasing level of certainty about the quality of the teachers, including 
sessional staff, and the organisation’s quality statement overall. Placing a focus on 
learning and teaching quality indirectly affects the student experience. It does not 
have to be mandatory, as academics can choose to undertake summative observations 
as part of a promotion application. 

On the other hand, PRoT creates another level of procedure to be maintained and 
another set of policies relating to the use of peer observation for promotion purposes; 
hence it is a further impost from above and demands more from academics who are 
already meeting increased workloads without compensation. 

Participants 
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PRoT has a number of advantages and disadvantages for participants. In terms of the 
negatives, the summative model of PRoT can be diminished from the academic 
participant’s perspective if participation is not tied to some official recognition of 
professional development (such as a qualification) and a developmental plan to help. 
Professional-development opportunities would help academics understand the criteria 
used for observation and how to achieve them. Participants may perceive PRoT as an 
inspectorial model that discourages intrinsic desire for participation. Participants may 
be nervous about “failure”. Participants may have questions about whether the 
reviewer is both qualified and trustworthy. Participants may have questions about 
what can be reviewed (pedagogy, curriculum design etc.). Thus if PRoT was applied 
to a range of different learning and teaching operations, rather than just face-to-face 
teaching, it would be more advantageous for participants. Participants may have 
questions about the choice of reviewer; hence having more than one observer for 
summative form (as in PRoT of research publications) and student evaluations 
ensures a moderated set of comments and data that can be triangulated. The 
summative model has the potential to create power distance between colleagues. 

In positive terms, PRoT is a process that participants can follow to gain independent 
evidence for promotion, confirmation and awards. PRoT provides a clear set of 
teaching and learning criteria that can be assessed against and prepared towards. It 
makes no difference whether PRoT is voluntary or compulsory as long as there is 
intrinsic motivation to participate. It requires engagement with learning and teaching 
quality issues and with developmental processes. If the developmental PRoT process 
can be linked to the summative process in terms of preparation, this will encourage 
participation. Having an observation report for the evaluation provides key feedback 
for development and encourages “buy-in”. The developmental model, as opposed to a 
summative model, can create greater collaboration between colleagues. 

Conclusion 

This article has painted an accurate and detailed picture of the publicly visible aspect 
of the PRoT landscape in Australia. The limitation of this review is that it is a 
snapshot; with the growing quality agenda provided by TEQSA and competition 
between universities for students, quality assurance is most likely to be a dynamic and 


21 


21 



Journal of University Teaching & Learning Practice, Vol. 12 [2019], Iss. 1, Art. 3 


changing landscape. The nature of the instruments and of the organisational context 
and processes that use them have been identified. The conclusion is that we have 
provided a landscape description of the use of peer observation of teaching in the 
Australian university context. It is limited to information promoted via their public 
websites. We have provided a framework by which to review this landscape and 
enriched the picture of aspects of the questions in our framework by adding evidence 
collected in a case study of experiences of PRoT protocols. This evidence provides 
critical insight into potential problems and solutions for creating effective systems. 

This article reviewed the process of peer review at one institution in Queensland 
through the eyes of the participants. Motivations behind participation in the peer- 
review process through participant reflections have been identified. In addition, 
success factors have been identified. The results of the study indicate that the peer- 
review process has the potential to not only significantly affect academics’ pedagogy 
but to improve teaching confidence and associated benefits in regard to evidence- 
based teaching for promotional opportunities. The benefits of the peer-review process 
extend to reviewee and reviewer. 

The experiences described here validate some aspects of previous work in relation to 
the motivations for participating in peer review. In some ways this study expands on it 
by describing the process from multiple participants. In this way it offers some new 
perspectives on peer review, which should serve to motivate academics to participate 
in what has been regarded as a “policing” policy. The most significant aspect of the 
process reported here is the concept of peer review being conducted in a “triad”, as 
peer review is often undertaken on a one-to-one basis in universities. The triad 
structure offers additional perspectives that a pair does not; hence, it is an 
improvement due to the increased rigour, trustworthiness and validity it gives to the 
PRoT process. 
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